Towards Automation in Architectural Photogrammetry Using Digital Image Processing

This paper describes some approaches towards an automation in architectural photogrammetry using digital image processing. We present techniques for semi-automatic 3D measurement and the application of digital image analysis to support the extraction of facade primitives. The proposed semi-automatic 3D measurement is used for the determination of tie points for the orientation and the interactive restitution process. The use of digital image analysis in architectural photogrammetry allows the automation of various low-level tasks. Exemplary, we introduce some approaches for a computer assisted generation of a facade map from gray-level photographs. Some results for the mapping of joint contours, i.e. for irregular stones, are presented. 1. AUTOMATION IN ARCHITECTURAL PHOTOGRAMMETRY Architectural photogrammetry is well suited to extract information from images, required for documentation, restoration and reconstruction. During the last three decades, there have been enormous improvements in the analytical photogrammetry, providing much better orientation techniques. Contrary, quite a few improvements can be notified in the restitution process. For example, the sequential measurement of corresponding points in two or more digital images is a step back, compared to the traditional stereo restitution. In Architectural Photogrammetry two restitution techniques are widely used. The first is the stereophotogrammetric approach. Images are acquired in a stereo constellation with nearly parallel viewing directions straight to the baseline. Only image pairs with a limited convergence, acquired straight in front of the facade with a base–distance relation between 0.1 and 1, are well suited for the restitution. Other arrangements obstruct the stereo view and the restitution. This arrangement is a limitation factor in many situations. Another shortcoming of this approach is the requirement for expensive hardware like analytical plotters. So far no low-cost and reliable hardand software is available for a digital stereophotogrammetric restitution for architectural purposes. Beside these shortcomings, stereophotogrammetry delivers precise and reliable results. If a large amount of data is required the expenditures are justified (DALLAS, 1995). The other widely used approach is the restitution based on bundle adjustment. A lot of low-cost systems have been developed for these purposes (FELLBAUM, 1992). Whereas in older systems usually digitizer have been used for the measurement of image coordinates, today the common technique is the measurement on the screen in the scanned images. The main shortcoming of the bundle oriented systems for orientation and for rectification is the necessity to measure points sequentially in the different images. Therefore only clearly identifiable points can be used. In some systems epipolar lines are displayed in the images after the orientation process. This provides the operator with the opportunity to measure points on edges intersected by the epipolar lines. Until now, the operator has to switch between the images and to re-identify the points. This procedure is time consuming and produces a lot of blunders. This blunders are difficult to detect and to solve, because of the separation of measurement and calculation. With modern computers and software concepts like multithreading it should be possible to execute real time orientation calculations in the background during the interactive measurement, even on PCs. This would inform the operator directly after the measurement about the influence of the new measurement on the whole orientation. But the main emphasis of this paper is put on the restitution process, which requires an enormous amount of interactive work. The greatest part is very tedious and wastes the high capacity of the human operator, e.g. the monotonous mapping of joint contours or the acquisition of window frames. On the other hand, the interactive restitution in architectural photogrammetry requires high level interpretation, a lot of a priori knowledge and specific experience, i.e. the identification of building styles and the acquisition of characteristic shapes. Due to the complexity of these tasks and the lack of solutions offered by classical pattern recognition and image analysis, an automation of this specialized human work is difficult. So far in many software packages the measurement of reseau crosses is the only task, where digital image processing is used for automation. But the wide varieties of features at building prevent that full automatic image analysis techniques can be used in architectural photogrammetry. Therefore, we decided to follow two independent developments to avoid these problems. The first is to use semi-automatic techniques, which support the operator – but leave the decision for each feature to extract in the hand of the operator. This requires not notably more work than automatic processes followed by interactive control. The second approach is to confine oneself to a few but frequent features. 2. SEMI-AUTOMATIC MEASUREMENT Special attention is put on the possibilities to support the interactive point measurements by digital image processing techniques, like matching. This paper describes the semi-automatic measurement of identical points for the bundle adjustment and for the restitution process. For these purposes matching techniques are used, but they need sufficient initial values that are provided by various geometric constraints. 2.1. Measurement of Tie Points for the Orientation For relative orientation and bundle adjustment a large amount of tie points is required. The more and better distributed tie points are available the better results can be anticipated. Matching techniques are well suited for the measurement of tie points. A point in the first image is defined by its position in the image matrix. An area around the point, e.g. 15 x 15 pixels, is defined as reference matrix. In the second image a larger area around the anticipated position of the corresponding point is defined. Between the reference matrix and the corresponding matrix the correlation coefficient is calculated within the search area. The corresponding point is anticipated at the position with the maximum correlation coefficient. There are further techniques available for the measurement of tie points, like leastsquares matching (WEWEL, 1996) or feature based matching (FÖRSTNER & GÜLCH, 1987). We examine these techniques for application in architectural photogrammetry. All this approaches need initial values for the definition of a suitable search area. In aerial photogrammetry initial values for the definition of the search matrix can be derived from the parameters of the photo flight. Because of the very different viewing angles on the same point, it is necessary to inform the operator about the result of the matching process and wait for the decision of the operator, whether he accepts the result or not. 2.2. Measurement for the Restitution If orientation data are available, it is possible to reduce the search area of a correlation process. The Vertical Line Locus (VLL) technique reduces the search area of matching processes in aerial images by defining a vertical line in the object space and calculating the correlation coefficients of the two image matrices according to a point along this line (COGAN & HUNTER, 1984). Adapting this for architectural approaches is difficult, because no horizontal line is suited to replace the vertical line in any arrangement. Another approach is to define a master image, in which most of the restitution work can be done. To acquire the 3D coordinates of a point, it has to be measured interactively in the master image. A correlation process along the epipolar line searches the corresponding points in one or more slave images (Fig. 2). Resulting from a reduced search area the probability of failures is significantly reduced and the process is accelerated. This relieves the operator from the time consuming sequential measurement of points in subsequent images. After each change of the image on the screen the operator has to search the cursor in the image. A lot of problems in architectural photogrammetry result from a confusion of similar looking object points. The eyes of the operator stay on the master image if the Number of already measured points Prediction technique